Neuroevolutionary reinforcement learning for generalized control of simulated helicopters

This article presents an extended case study in the application of neuroevolution to generalized simulated helicopter hovering, an important challenge problem for reinforcement learning. While neuroevolution is well suited to coping with the domain’s complex transition dynamics and high-dimensional state and action spaces, the need to explore efficiently and learn on-line poses unusual challenges. We propose and evaluate several methods for three increasingly challenging variations of the task, including the method that won first place in the 2008 Reinforcement Learning Competition. The results demonstrate that (1) neuroevolution can be effective for complex on-line reinforcement learning tasks such as generalized helicopter hovering, (2) neuroevolution excels at finding effective helicopter hovering policies but not at learning helicopter models, (3) due to the difficulty of learning reliable models, model-based approaches to helicopter hovering are feasible only when domain expertise is available to aid the design of a suitable model representation and (4) recent advances in efficient resampling can enable neuroevolution to tackle more aggressively generalized reinforcement learning tasks

The European 2015 drought from a hydrological perspective

In 2015 large parts of Europe were affected by drought. In this paper, we analyze the hydrological footprint (dynamic development over space and time) of the drought of 2015 in terms of both severity (magnitude) and spatial extent and compare it to the extreme drought of 2003. Analyses are based on a range of low flow and hydrological drought indices derived for about 800 streamflow records across Europe, collected in a community effort based on a common protocol. We compare the hydrological footprints of both events with the meteorological footprints, in order to learn from similarities and differences of both perspectives and to draw conclusions for drought management. The region affected by hydrological drought in 2015 differed somewhat from the drought of 2003, with its center located more towards eastern Europe. In terms of low flow magnitude, a region surrounding the Czech Republic was the most affected, with summer low flows that exhibited return intervals of 100 years and more. In terms of deficit volumes, the geographical center of the event was in southern Germany, where the drought lasted a particularly long time. A detailed spatial and temporal assessment of the 2015 event showed that the particular behavior in these regions was partly a result of diverging wetness preconditions in the studied catchments. Extreme droughts emerged where preconditions were particularly dry. In regions with wet preconditions, low flow events developed later and tended to be less severe. For both the 2003 and 2015 events, the onset of the hydrological drought was well correlated with the lowest flow recorded during the event (low flow magnitude), pointing towards a potential for early warning of the severity of streamflow drought. Time series of monthly drought indices (both streamflow- and climate-based indices) showed that meteorological and hydrological events developed differently in space and time, both in terms of extent and severity (magnitude). These results emphasize that drought is a hazard which leaves different footprints on the various components of the water cycle at different spatial and temporal scales. The difference in the dynamic development of meteorological and hydrological drought also implies that impacts on various water-use sectors and river ecology cannot be informed by climate indices alone. Thus, an assessment of drought impacts on water resources requires hydrological data in addition to drought indices based solely on climate data. The transboundary scale of the event also suggests that additional efforts need to be undertaken to make timely pan-European hydrological assessments more operational in the future

Combined use of relative drought indices to analyze climate change impact on meteorological and hydrological droughts in a Mediterranean basin

[EN] Standardized drought indices have been traditionally used to identify and assess droughts because of their simplicity and flexibility to compare the departure from normal conditions across regions at different timescales. Nevertheless, the statistical foundation of these indices assumes stationarity for certain aspects of the climatic variables, which could no longer be valid under climate change. This contribution provides a framework to analyze the impact of climate change on meteorological and hydrological droughts, considering shifts in precipitation and temperature, adapted to a Mediterranean basin. For this purpose, droughts are characterized through a combination of relative standardized indices: Standardized Precipitation Index (rSPI), Standardized Precipitation Evapotranspiration Index (rSPEI) and a Standardized Flow Index (rSFI). The uncertainty and the stationarity of the distribution parameters used to compute the drought indices are assessed by bootstrapping resampling techniques and overlapping coefficients. For the application of the approach to a semiarid Mediterranean basin (Jucar River Basin), the Thornthwaite scheme was modified to improve the representation of the intra-annual variation of the potential evapotranspiration and low flow simulation in hydrological modelling was improved for a better characterization of hydrological droughts. Results for the Jucar basin show a general increase in the intensity and magnitude of both meteorological and hydrological droughts under climate change scenarios, due to the combined effects of rainfall reduction and evapotranspiration increase. Although the indicators show similar values for the historical period, under climate change scenarios the rSPI could underestimate the severity of meteorological droughts by ignoring the role of temperature. (C) 2017 Elsevier B.V. All rights reserved.This study has been supported by the IMPADAPT project (CGL2013-48424-C2-1-R) with Spanish MINECO (Ministerio de Economia y Competitividad) and European FEDER funds. Patricia Marcos-Garcia is also supported by a FPI grant from the PhD Training Program (BES-2014-070490) of the former MINECO. The authors thank AEMET (Spanish Meteorological Office) and University of Cantabria for the data provided for this work (dataset Spain02).Marcos-García, P.; Lopez-Nicolas, A.; Pulido-Velazquez, M. (2017). Combined use of relative drought indices to analyze climate change impact on meteorological and hydrological droughts in a Mediterranean basin. Journal of Hydrology. 554:292-305. https://doi.org/10.1016/j.jhydrol.2017.09.028S29230555

A steady-state genetic algorithm with resampling for noisy inventory control

Noisy fitness functions occur in many practical applications of evolutionary computation. A standard technique for solving these problems is fitness resampling but this may be inefficient or need a large population, and combined with elitism it may overvalue chromosomes or reduce genetic diversity. We describe a simple new resampling technique called Greedy Average Sampling for steady-state genetic algorithms such as GENITOR. It requires an extra runtime parameter to be tuned, but does not need a large population or assumptions on noise distributions. In experiments on a well-known Inventory Control problem it performed a large number of samples on the best chromosomes yet only a small number on average, and was more effective than four other tested technique

Kinase activity of ArcB from Escherichia coli is subject to regulation by both ubiquinone and demethylmenaquinone

Expression of the catabolic network in Escherichia coli is predominantly regulated, via oxygen availability, by the two-component system ArcBA. It has been shown that the kinase activity of ArcB is controlled by the redox state of two critical pairs of cysteines in dimers of the ArcB sensory kinase. Among the cellular components that control the redox state of these cysteines of ArcB are the quinones from the cytoplasmic membrane of the cell, which function in 'respiratory' electron transfer. This study is an effort to understand how the redox state of the quinone pool(s) is sensed by the cell via the ArcB kinase. We report the relationship between growth, quinone content, ubiquinone redox state, the level of ArcA phosphorylation, and the level of ArcA-dependent gene expression, in a number of mutants of E. coli with specific alterations in their set of quinones, under a range of physiological conditions. Our results provide experimental evidence for a previously formulated hypothesis that not only ubiquinone, but also demethylmenaquinone, can inactivate kinase activity of ArcB. Also, in a mutant strain that only contains demethylmenaquinone, the extent of ArcA phosphorylation can be modulated by the oxygen supply rate, which shows that demethylmenaquinone can also inactivate ArcB in its oxidized form. Furthermore, in batch cultures of a strain that contains ubiquinone as its only quinone species, we observed that the ArcA phosphorylation level closely followed the redox state of the ubiquinone/ubiquinol pool, much more strictly than it does in the wild type strain. Therefore, at low rates of oxygen supply in the wild type strain, the activity of ArcB may be inhibited by demethylmenaquinone, in spite of the fact that the ubiquinones are present in the ubiquinol form

STED super-resolution microscopy reveals an array of MINOS clusters along human mitochondria.

The mitochondrial inner membrane organizing system (MINOS) is a conserved large hetero-oligomeric protein complex in the mitochondrial inner membrane, crucial for the maintenance of cristae morphology. MINOS has been suggested to represent the core of an extended protein network that controls mitochondrial function and structure, and has been linked to several human diseases. The spatial arrangement of MINOS within mitochondria is ill-defined, however. Using super-resolution stimulated emission depletion (STED) microscopy and immunogold electron microscopy, we determined the distribution of three known human MINOS subunits (mitofilin, MINOS1, and CHCHD3) in mammalian cells. Super-resolution microscopy revealed that all three subunits form similar clusters within mitochondria, and that MINOS is more abundant in mitochondria around the nucleus than in peripheral mitochondria. At the submitochondrial level, mitofilin, a core MINOS subunit, is preferentially localized at cristae junctions. In primary human fibroblasts, mitofilin labeling uncovered a regularly spaced pattern of clusters arranged in parallel to the cell growth surfaces. We suggest that this array of MINOS complexes might explain the observed phenomenon of largely horizontally arranged cristae junctions that connect the inner boundary membrane to lamellar cristae. The super-resolution images demonstrate an unexpectedly high level of regularity in the nanoscale distribution of the MINOS complex in human mitochondria, supporting an integrating role of MINOS in the structural organization of the organelle